1. Field of the Invention
The present invention generally relates to the field of functional verification of complex systems. More particularly, the present invention relates to methods, systems, and media for autonomous management and control of distributed verification of designs such as integrated circuit (IC) designs.
2. Description of the Related Art
When designing increasingly complex processors or IC chips such as Application-Specific ICs (ASICs) and system-on-chips (SoC's), functional verification has proven to be a major bottleneck in achieving time-to-market goals. Many companies now realize that functional verification of complex chips has become an inefficient, unwieldy, and incomplete process. Design teams report that functional verification of medium- to large-complexity processors, ASICs or SOC's may consume over 70% of the project's manpower, schedule and budget. In spite of the time and resources consumed by functional verification, it is still often incomplete, allowing design bugs to go undetected.
The design process for a Very Large Scale Integrated (VLSI) chip starts with the creation of a functional specification for the design. Once the functional specification has been completed, the verification team typically creates a test plan that specifies the design features and functions to be tested at both the block and system levels. The verification team then creates testbenches (also known as tests) such as deterministic tests and tests automatically generated to verify design functionality until all test plan requirements have been met. The process of verifying the functional specification is called functional verification, which encompasses the development and implementation of this test plan.
Functional verification ensures functional conformance of an integrated circuit design to its architectural and microarchitectural specifications and determines whether a design is logically correct. The verification process involves developing and simulating tests that are used to determine whether design components (e.g., processor units, resources, functions, etc.) behave according to their functional specification, from both an architectural and microarchitectural perspective. Functional verification is typically completed before fabrication of the processor, as finding and fixing errors, or bugs, after fabrication proves to be time-consuming and expensive.
Today, the scope of functional verification has gone beyond simulation. While simulation is still the major component of functional verification, other methods have enhanced the ability to ensure the correctness of the design. The tools created to augment the simulation utilizing these methods, however, are generally of limited utility from one project to another. These tools must therefore be customized for use in the functional verification of a particular processor or for a specific project. Personnel skilled in customizing these generic tools have become a valuable commodity, as verification teams that can write testcases, behaviorals, and checkers that efficiently isolate the bugs in a design have a major, positive influence on time-to-market and costs.
To manage the verification of today's complex designs and to be responsive to frequent design specification changes and upgrades, an efficient, iterative process that may be used throughout the duration of the design is desired. Hardware and software teams have consistently found that iteration is an inevitable part of the design process. There is significant value in planning for iteration and developing a methodology that minimizes the overall design time. Verification is a major component of the design process and efficient management of its tasks and resources are important in reducing the number of design iterations and optimizing the length of each iteration. Improved functional verification can cut costs, improve design quality and accelerate time-to-market. Moreover, improved functional verification enables companies to sharply increase the productivity of their precious resource, verification personnel.
Different methods have been developed to improve the quality and efficiency of the functional verification. These methods include random test generation, coverage-driven verification, coverage-driven test generation, formal verification methods and assertion-based verification.
Random test verification is the attempt to randomly generate tests to validate some feature of the design. Random test verification, while it may appear to reduce the effort necessary for test generation, instead typically generates low quality and redundant tests that may not cover all features of the design. Generation, simulation, and debugging of low quality and redundant tests wastes scarce verification resources.
Coverage Driven Verification (CDV) is the integrated verification methodology and environment where verification tasks and activities are defined and scheduled based on the actual verification achieved thus far (i.e. actual functional coverage). Functional coverage is the study and analysis of the quality and completeness of functional verification. It includes statistical, stochastic and heuristic analysis of functional verification progress and completeness. CDV is independent of any specific tool or method but rather utilizes any and all available tools and method to achieve a pre-defined set of verification goals. CDV is heavily dependent on a comprehensive and accurate coverage analysis.
Coverage Driven Generation (CDG) refers to test generation (Random, Psuedo-Random, Deterministic, etc.) with an awareness of functional coverage achieved by tests generated thus far. CDG may also refer to the act of test generation itself, where rather than specifying test specifications, the user specifies functional coverage attributes desired. The test generator then tries to generate a set of tests that target the desired coverage attributes. Coverage analysis techniques are utilized for estimating or measuring the architectural and microarchitectural attributes of a generated test.
Formal methods can be applied to the design for verification of design features, as well as identifying design connectivity and signal propagation characteristics. In most such cases, a set of formal rules or design behavior specifications are defined and applied against a compiled model of the design to identify any deviation from the formal specification (rules). Some tools can generate a set of checkers and monitors from the formal rules which can then be used along with the simulation-based verification.
Assertion-based verification technique often relies on arbitrary or non-test plan based factors. Design and verification engineers define checkers and monitors that define design features that need to be verified or critical sequences of operations to be monitored. There is no way to know that all necessary checkers and monitors have been defined. Assertion-based verification is flawed by the inability to have any certainty that functional verification is complete, but it does pinpoint the source of a malfunction when one is detected.
Functional verification systems must measure functional coverage, identify holes in functional verification, and determine with confidence when functional verification is complete before committing to fabrication. Unfortunately, current functional verification approaches and their limitations cause many design teams to rely on code coverage, rate of new bugs found, or the project schedule to determine when functional verification is completed.
Therefore, there is a need for methods, systems, and media to reduce the amount of resources required for functional verification, to improve its quality, and to provide more effective means for managing distributed and heterogeneous verification environments. Continuous and flexible monitoring of verification processes to determine when verification goals have been achieved is also desired.